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Abstract — Provable lower bounds are presented for the infor- 
mation rate I{X;X + S + N) where X is the symbol drawn 
independently and uniformly from a finite-size alphabet, S is 
a discrete-valued random variable (RV) and TV is a Gaussian 
RV. It is well known that with S representing the precursor 
intersymbol interference (ISI) at the decision feedback equalizer 
, (DFE) output, I{X\ X + S + N) serves as a tight lower bound 

< for the symmetric information rate (SIR) as well as capacity of 
the ISI channel corrupted by Gaussian noise. When evaluated on 
a number of well-known flnite-ISI channels, these new bounds 
provide a very similar level of tightness against the SIR to the 
conjectured lower bound by Shamai and Laroia at all signal- 
to-noise ratio (SNR) ranges, while being actually tighter when 
viewed closed up at high SNRs. The new lower bounds are 
obtained in two steps: First, a "mismatched" mutual information 
function is introduced which can be proved as a lower bound 
to I{X; X + S + N). Secondly, this function is further bounded 

I from below by an expression that can be computed easily via a 
, few single-dimensional integrations with a small computational 

< load. 

Index Terms — Channel capacity, decision feedback equalizer, 
information rate, intersymbol interference, lower bounds, mutual 
information. 



I. Introduction 

The computation of the symmetric information rate (SIR) 
of the classical discrete-time intersymbol interference (ISI) 
channel is of great interest in digital communication. The SIR 
represents the mutual information between the channel input 
and output while the input is constrained to be independently 
and uniformly distributed (i.u.d.) over the given alphabet. In 
this sense, the SIR is also known as capacity with uniform, 
independent input distribution and itself represents a reason- 
ably tight lower bound to unconstrained channel capacity, 
especially at high coding rates. During recent years, a number 
of researchers have worked on estimating or bounding the 
information rate via simulation of the Bahl-Cocke-Jelinek- 
Raviv (BCJR) algorithm fll. The information rate with a 
given input distribution can be closely estimated for finite 
ISI channels with moderate input alphabet size and channel 
impulse response length, by running the forward-recursion 
portion of the BCJR algorithm on long (pseudo) randomly 
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generated input and noise samples f2], f3], f4l. The simulation- 
based method has been further generalized, and lower and 
upper bounds based on auxiliary finite-state channels with 
reduced states were introduced for long ISI channels, as well 
as some non-finite state ISI channels in [5|. The tightness of 
these bounds is highly related to the optimality of auxiliary 
channels, but the general rule to find the optimal or near- 
optimal auxiliary channel has not been provided in jSl. The 
work of in has been recently extended in ||6l to further tighten 
the lower and upper bounds by using an iterative expectation- 
maximization type algorithm to optimize the parameters of the 
auxiliary finite-state channels. It is noted, however, that the 
global optimality of the bounds in |j6l is neither guaranteed, 
nor the lower bound is proven to converge to a stationary 
point as iteration progresses. Another approach based on 
auxiliary channels is also proposed to obtain a lower bound 
utilizing a mismatched Ungerboeck-type channel response to 
achieve improved tightness for a given level of computational 
complexity 121 . In the context of the Ungerboeck-type 
response is the channel's response observed at the output 
of the matched filter front-end. As such, the trellis search 
detection algorithms driven by the channel observations of the 
Ungerboeck model must be designed so that they can handle 
correlated noise samples lHJ. 

An entirely different direction in estimating or bounding the 
information rate is based on finding an analytical expression 
that can easily be evaluated or numerically computed (in 
contrast to the methods based on Monte-Carlo simulation that 
rely on generating pseudo-random signal and noise samples). 
An early work in this direction is the lower bound on the 
SIR by Hirt [9| based on carving a fixed block out of the 
channel input/output sequences and performing a single multi- 
dimensional integration (or running Monte-Carlo simulation 
for estimating the integral) with the dimensionality equal to 
the block size. However, this method is also computationally 
intense unless the size of the block gets small. Unfortunately 
the lower bound of |9| is not tight unless the block size is 
very large compared to the channel ISI length. 

A number of more computationally efficient and analytically 
evaluated lower bounds for the SIR have been discussed in 
ifTOl . ifTTI . Unfortunately, however, the only bound presented 
in ifTTI that is reasonably tight throughout the entire signal- 
to-noise ratio (SNR) region (i.e., both low and high code 
rate regimes) is the one that could not be proved as a lower 
bound. This particular bound is now widely known as the 
Shamai-Laroia conjecture (SLC) and, although unproven, is 
a popular tool for quickly estimating the SIR of ISI channels. 
At high code rates, the SIR is generally very close to capacity. 
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SO an easily computed tight SIR lower bound is also useful 
for quickly estimating channel capacity for high code rate 
applications, such as data storage channels and optical fiber 
channels. 

Consider the random variable (RV) Y ~ X + S + N, where 
X is a symbol drawn independently and uniformly from a 
fixed, finite-size alphabet set symmetrically positioned around 
the origin, S a zero-mean discrete-valued RV, and N a zero- 
mean Gaussian RV. The SLC is concerned with the special 
case where 5* is a linear sum of symbols drawn independently 
and uniformly from the same symbol set where X was taken. 
As the number of symbols forming S grows, finding an 
analytical expression for the probability density function of 
S + N (and thus one for I{X; Y)) is a long-standing problem 
lfT3l . [[I4|, as pointed out in [ 1 1 1. The SLC of 1 1 1 1 can be stated 
as I{X; X + S + N)> I{X; X + G), where G is a Gaussian 
RV with variance matching that of S + N. The information 
rate I{X; X + G) is easily obtained by numerically calculating 
a single one-dimensional integral, and is generally observed 
to be reasonably tight to I{X;X + S + N) in most cases. 
Unfortunately, I{X;X + G) remains as a conjectured bound 
with no proof available to date. One difficulty of proving 
the SLC stems from the fact that for the channels driven by 
the inputs from a finite alphabet, Gaussian noise is not the 
worst-case noise in terms of the achievable information rate 
ifm . IIT2I . Another difficulty is that the power contribution of 
a single individual weight involved in constructing 5* could 
remain a significant portion of the total power associated with 
all weights, even if the number of weights approaches infinity. 
This is to say that the Lindberg condition for the central limit 
theorem does not hold for this problem, and the Gaussian 
approximation of S cannot be justified ifTTl . 

In this paper, we are also interested in the easily computable 
analytical expressions for lower bounds to I{X; X + S + N). 
Note that, in the context of the unbiased minimum mean- 
squared-error decision feedback equalizer (MMSE-DFE) ap- 
plication, S represents the collection of residual precursor ISI 
contributions and in this case I{X;X + S + N) itself is a 
well-known lower bound to the SIR [11 J. The bounds we 
develop here are fairly tight, with their tightness generally 
enhanced with increasing computational load (which in the 
end still remains small). Our approach is to first define a 
"mismatched" mutual information (MI) function based on the 
"mismatched" entropy that takes the log operation not on 
the actual underlying probability density but on the Gaussian 
density with the same variance. We then prove that this 
"mismatched" MI is always less than or equal to I{X; Y). We 
further bound this function from below so that the final bound 
can be evaluated using numerical integration. The bound is 
basically evaluated by computing a few single-dimensional 
integrals. This is in contrast to the Hirt bound that computes 
a single multi-dimensional integral of very high dimension. 
Our bound computation also requires the evaluation of sum 
of the absolute values of the linear coefficients that form S 
as well as the identification of dominant coefficient values, 
if they exist. With the application of the MMSE-DFE, these 
linear coefficients correspond to the weights on the interfer- 
ing symbols after ideal postcursor ISI cancellation and can 



easily be obtained with a small amount of computation. At a 
reasonable overall computational load, our bounds are shown 
to be for all practical purposes as tight as the Shamai-Laroia 
conjecture for many practical ISI channels. 

Section HI] presents the provable bound to I{X;Y) and 
numerically compares it with the SLC for some example 
distributions for the linear coefficients that form S. Section 
Hin develops upper and lower bounds on the provable bound 
itself, based on identifying clusters in the distribution of 
S + N. Finding clusters in the S + N distribution is the 
same as identifying dominant coefficient values from the linear 
coefficient set that is used to construct S. Section HVl generates 
and discusses numerical results. In all finite-ISI channels 
examined, our bound provides the same level of tightness as 
the SLC against the SIR (while being actually tighter than 
SLC at high SNRs when viewed closed up) with a very 
reasonable computation load. In particular, our lower bound is 
presented on the same channel employed in |6|. This provides 
an indirect means to compare the computational loads of our 
method and that of 16|. As expected, our analytical method 
is considerably better in quickly producing a reasonably tight 
bound than the simulation-based method of [6] in terms of 
complexity/accuracy tradeoffs. Note that the method of f6\ 
represents the latest development in simulation-based SIR 
bounds. Section IV] concludes the paper. 

II. A Provable Lower Bound to the Symmetrical 
Information Rate 

We first present a provable lower bound to I{X; Y) where 
Y = X + J2k=i d-kXk + N ^X + S + N.Th& symbols X 
and Xk are all independently and uniformly drawn. The linear 
coefficients d-^'s are related to the channel impulse response 
and will be specified in Section |IV] Let F = S* + iV so we 
can write Y = X + V . Note that y is a Gaussian mixture. 
Also let Z — X + G where G is a zero mean Gaussian with 
variance matching that of V, i.e., = ay. 

Definition 1 ("Mismatched" MI (MMI) Function): Define 

r{X;Y)^H'{Y)- H'{V) (1) 

where 

/oo 
frit) log fz{t)dt, 
-00 

/OO 
fv{t) log fG{t)dt 
-oo 

and fv{t), fz{t), and /g(0 are the probability density 

functions (pdfs) of the RVs, Y, V, Z, and G, respectively. Note 
that the "mismatched" entropy functions H'iY) and H'{V) 
are defined based the log operation applied not to the actual 
underlying pdf fv{t) but rather to the "mismatched" Gaussian 
pdf fait). 

Lemma 1: Given the MMI function defined as above, we 
have 

I'{X;Y) <I{X;Y). (2) 

Proof: See Appendix lAl ■ 
Let us now take a close look at this MMI function I'{X\ Y) 
and develop some insights into its behavior Let the variances 
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of V, S, and N be ay, (t|, and cr^ respectively. Further as- 
sume that the RVs, X, V, S, and N are all real-valued. We will 
also assume a binary input alphabet. These assumptions are 
not necessary for our development but make the presentation 
clearer as well as less cluttered. We will simply state the results 
in Section ITlI-Cl for a non-binary/complex-valued example. We 
also denote nii = X]fc=i d-kXk for i = 1,2, ... ,2^ since 
{Xk}^^i can have 2^ different sequences. Naturally, the pdfs 
of RVs V and G can be written as 



fv(t) 



fait) 



2^ 



1 



^ ^/2^Ta 



GXp 



2< 



V^7 



: exp 



Proposition 1: Denoting pi = rrii j \JPx and r = (i — 
'rfii)la]^, letting p^'s to mean the positive-half subset of p^'s, 
and defining R = Px / and = cttv / '^v^ the MMI function 
can be rewritten as V {X;Y^ = log 2 — F with the new 
definition 



F^2 



^Ee. [log{ 

i=\ 

r [log{l + e- 



1 



2_Rpg-20VflT 



-2_R 



2I.-1 

E 

fc=i 



+e 



ilog|l + 2cosh(2i?p^ 



(3fl) 



-2cI>^/Rt-2R 



.g|l + 2cosh(2i?p^ 



-20V^T-2_R 



(3^7) 



A detailed derivation is given in Appendix |B] The po- 
sition nii of the ith Gaussian pdf of the mixture is 
expressed as a dimensionless quantity: pi — rrii/^/Px, with 
the normalization by the square root of the input power. 
Because of the symmetric nature of fv{t), pi occurs in 
equal-magnitude, opposite-polarity pairs. The expectation is 
initially over r, which is considered a zero-mean unit-variance 
Gaussian random variable when contained inside the argument 
of the expectation operator. The expectation operator in this 
case can simply be viewed as a short-hand notation as in 



V2 



■p(T)dT. 



Er [p{r)] 



In dSol l and (l3fcl ). however, p (or p+) is also treated as a 
RV and the expectation is over both r and p (or r and 
p+) as the double subscripts indicate. Given the pdfs of t, 
p and p+, the computation of the expectation now involves 
numerical evaluation of a double integral. Note that in ( l3a] l 
p is a discrete-valued random variable distributed according 
to fp{t), which denotes the probability distribution of p = 
i^/ VPx)^k=i'^-k^k ™d p+ is a discrete- valued random 
variable distributed according to 2fp{t)u{t) where u{t) is a 



step function. Also, notice that cosh(2i?p+) > 1 and < 1. 
Since it is not easy to find fp{t) when L is large, evaluating 
(|3a] l or ( [3fc] l is difficult in general. 
It is insightful to compare F with 



FsLC = log 2 



- CsLc{R) 



:E, 



log{l 



-2VRt-2R 



}] 



log n + 2e 



-4Vr.t-4:R\ 



(4a) 



(4b) 



where Cslc{R) is the SIR of the binary-input Gaussian chan 

4 



nel with SNR given by i? = Px jdy and is the well-known 



SLC. The function Fg^c quantifies the gap between the SLC 
and the maximum attainable capacity for any binary channel 
with infinite SNR, namely, 1 bit/channel use. Comparing the 
expressions for F in ( [3fcl l and Fgi^c in (|43] i, we see that if 
p+ = so that 1, then F = Fslc, and V [X;Y) 

and the SLC both become equal to 1(X; y). Also, if the 
discrete RV p converges to a Gaussian random variable (in 
cumulative distribution), then again we get F ~ Fslc and 
r{X;Y)^CsLc{R)^I{X;Y). 

Furthermore, that p+ > in ( |3^ makes F larger while the 
factor (j) being less than 1 has an effect of decreasing F as 
it increases. If I'{X;Y) — log 2 — F is to be a tight lower 
bound to I{X;Y), then F needs to be small. The important 
question is: how does F overall compare with Fslc, over 
all interested range of SNR? Since it is akeady proved that 
r{X; Y) = log 2 - i^, if F < Fslc for some R values, then 
clearly Cslc{R) = log 2 - Fslc < I{X; Y) at those SNRs, 
i.e., the SLC holds true at least at these SNRs. 

While exact computation of dSfcl l requires in gen- 
eral obtaining all possible positive-side values of p = 
(l/V-Px) X)fe=i d-kXk and thus can be computationally in- 
tense for large L, in the cases where we know the functional 
form of the distribution for p, evaluation of OoT l or i3b[ is 
easy; the behavior of F under different p distributions offers 
useful insights. 

First try a uniform distribution for p. For a uniformly 
distributed discrete random variable p from —KA — — |p|max 
to KA = |p|max with a gap A between delta functions in the 
pdf, we have 



K 



which makes 



2PxA^ V- .2 _ PxA^K{K + 1) 



2K+1 ^ 3 

i—1 

^x|p|max(|p|max + A) 





3 










+ 4 




= 1 - 


-R|p|max 


(|p|max - 





RA^K{K + 1) 



Fig.[T]shows F and Fslc plotted with K — 1000 as functions 
of R for various values of (j). We also consider a simple case 
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Fig. 1; F ~ FsLC as a function of R for a uniform p. 
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Fig. 2: F — FsLC as a function of R for a two-valued p. 



involving only a single coefficient d-i, in which case p takes 
only two possible values, e.g., p — ±-^(1 — (j)'^)/R. The plots 
of F and Fslc for this case are shown against R for different 
values of 4> in Fig. |2] Figs. [T] and |2]point to similar behaviors 
of F versus Fslc- Namely, F becomes smaller than Fslc 
as (j> decreases for a range of R values. At these R values, the 
provable lower bound I'{X; Y) is apparently tighter than the 
SLC, with respect to the SIR. 



III. Bounding F 

Exact computation of F in general is not easy, especially 
when L goes to infinity. We thus resort to bounding F with 
expressions that can easily be computed. An upper bound on F 
will provide a lower bound on I'{X; Y) and thus on I{X; Y). 
Lower bounds on F are also derived to see if they can get 
smaller than Fslc- If so, this would mean I'{X\ Y) = log 2 — 
F is larger than Cslc{R) = log 2 — Fslc^ i-^-, our bound is 
tighter than the SLC. 



A. Simple Bounds 

Since log f 1 + 2 cosh(2i?p+)e-2<^^^ 



is convex in p+, its integral function with respect to r. 



ilog (l + 2cosh(2i?p+)e 



also convex in p+. Moreover, this function increases as p+ 
increases. Accordingly, we can develop bounds on F. The 
first simple upper bound is 



(5) 



where, for a given |p|max, the function T (|p|max, d) represents 
a straight line passing through two points of the function 



i log ( 1 -h 2 cosh{2Re)e-'^'t'^'' + e-4</'v^r-4fl 



and at 

^L 



Note that \p\^ 



at « = 
max|pi| = 



-4fl 



IS 



X]fc=i M-fcl and CTp is the standard deviation of RV p. 

Similarly, E^ \ log + 2ae~2'^^^ + e""-^^^ 

a concave and increasing function of a = cosh(2i?(0+). Based 
on this property, we can develop another upper bound. 



F' 



■u2 A 



E. 



^log{ 



1 + 2(scrp + l)e 



-24>\/R.t-2B. 



(6) 



where s = (cosh(2i?|p|„iax) - 1) /|p|max, the slope 
of a straight line connecting two points (0, 1) and 

V I P I max 1 cosh(2i?|p|„iax))- 

A lower bound on F can also be obtained that can help 
shed lights on how tight the upper bounds on F are. Using 



the convexity of Er 
simple lower bound 



log \ \ + e-2-RPg-20Vl?r-2fl 

"of F is 



in (0, the 



ilog{l + 2e-20^^ 



-2R 



-40\/flT-4i?. I 



(7) 



Detailed derivations of (lU, and (|7]) are given in Appendix 

El 

B. Tightened Bounds Based on Cluster Identification 

The above bounds can be tightened up by identifying 
clusters in the Gaussian mixture fv{t)- In practical ISI chan- 
nels, fv{t) often consists of clusters. This is due to the 
fact that the coefficient set d-^'s typically contains a few 
dominating coefficients plus many small terms. Assuming 
there are Af dominating coefficients among d-fc's, we can let 
Pk = Xn+ Ih where 7i = 1, 2, . . . , 2*^ z = 1, 2, . . . , 2^-^^, 
and k = [n- 1)2^^^^ + i. Since Xk is an i.u.d. RV, A and 
p are independent so that cr^ = cr| + where cr^ and cr^ 
denote the variance of RVs A and p, respectively. Notice that 
A„ can be viewed as the position of a specific cluster while 
Pi points to a specific Gaussian pdf out of 2^"*^ Gaussian 
pdf's symmetrically positioned around A,i. 

Therefore, assuming there are 2*^ clusters of Gaussian pdfs, 
the upper bound i^"^ can be tightened as 



-^M — ^ ^ ^ Tn ( I I max : 1 

n=l 



(8) 
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where, for ci given |/i|max5 ^he function 7n(|M|niax; ^) is a 
straight Une that passes through the two points of the convex 
function i log |l + 2 cosh {2Re) p-^^^^e-^'t'^'^-^^ + 
g-4flA„g-40x/l?r-4J?,| at 61 = and 61 = |/i|max, <J^, is the 
standard deviation of RV fi defined as cr^ = ^ — a\, and 

I M I max — I P I max | | max ■ 

Another form of tightened upper bound based on F"^ is 
obtained as 



^ 2-^' ^ _ log |n- 2 (sMa^ + 1) < 



n=l 



-2RX„ 



(9) 



where Sm = (C0sh(2i?|/i|max) - 1) /l^lmax- 

The lower bound can also be tightened similarly based 
on the cluster identification: 

2M-1 

^|^A2-(M-i) ^ ilog{l + 2cosh(2i?A+) 



k=l 



where AjJ''s form the positive-half subset of A„'s. Detail 
derivations of ([8]l, (|9|l, and (fTOl i can be found in Appendix 



C. Bounds for Complex Channels with the Quaternary Alpha- 
bet Inputs 

In the previous subsections, ISI coefficients and noise sam- 
ples are assumed to be real-valued with the channel inputs 
being the binary phase shift keying (BPSK) signal. In this 
subsection, we provide a complex-valued example along with 
the channel inputs taken from a quadrature ph ase shift keying 

(QPSK) quaternary alphabet, i.e., Xk G ^^J^{±1 ± j)}- 
The extension to larger alphabets should be straightforward. 

Denoting the real and imaginary parts of complex number 
a by a^'') and a*-*^ respectively, i.e., a — a*^'"'' + ja^'^ and 
TTii — X]fc=i d-k^k for i = 1,2,..., 4^, the pdf's of complex 
random variables V and G are given as 



1 / (tW-mfy^ 



exp 



1 / (t( 

X ; — ^ exp - ^ 



N 



'N 



1 



: exp 



01 



1 / (tW) 
exp I -2- 

V 



Then, for the SLC, we write 

Fsic = log 4 -C5Lc(i?) 

/•OO 

:2 



/OO — T- 
-OO Vl" 



1 + e-2v^-2fl| rf^ 



= 2E^ 
= 2E^ 



log{ 



1 + e 



(llfl) 



i log {l + 2e-2^--2« + e-4v^r-4fl| 

(11^7) 



where E,- [p(t)] = tt ^/^e '^^p{T)dT. 
The function F is given as 



(12a) 



F ^ 4-^ ^ ( E, [log { 1 + e^^^^p'"' e-2^^--2fl| 
i=i ^ 

+E, [log {l + e-2y2flPr'e-20v^r-2fi|j ^ 
= 2E^<,, [log { 1 + e-2v^^p"-' e-20^--2i? j" 

4^-1 n 

^4-(L-i) ^ 2E, ilog{l + 2cosh(2V2i?pi'-)+) 

k=l L 

^g-20y2flr-27?, ^ g-40y2fir-4_R| 

i log {l + 2 cosh (2\/2i?p('-)+)e-20y2flr-2ii 



- 2Ep(.)+^^ 



^-40V2flr- 



r-4fl| 



(12/7) 



where pl 



ml VV-Px, Pl = "^^ 7 a/Fx, and s 



and p],' 's denote the positive-half subset of pi s and pi ''s 
respectively. The equality (|12a| ) holds because the pdf of p*^*") 
is identical to the pdf of p^^\ 

Then, the upper bound based on F"^ can be derived in a 
similar way as 



(13) 



"v/2 



where, for a given [p^'^^lmax, ^i''-' ( | | max, denotes a 
straight line that passes through the two points of the function 

e/ 



ilog|l + 2cosh(2V2i?0)e-2V2KA<:)g-20v^r 



-2R 



g-4\/2flAj^''' g-40V2KT-4_R 



at 6* = and at 6* = |Ai^''^|max- 

Note that |p'''''|max = |/i|max/\/2 and the variance of pS^^ is 
equal to cr^/2 since the pdfs of A^'"' and /i'^''' are identical to 
the pdfs of A^'-* and p^'^\ respectively. 
A second upper bound on F is given as 



^2Ejilog|l + 2 



(r) 

V2 



„^-20V2flT-2i?, , ^-4y2flA<''' ^-40V2KT-4i?, 
AC ~r c- fc- 



(14) 



where s^Q = (cosh(2y2i?|p('^)| 



1 /Im 
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Fig. 3: System Model of ISI channels. 



Finally, a lower bound to F can be shown to be 

Fi, ^ 4-*^ ^ i log { 1 + 2 cosh {2^R\t^+) 



Accordingly, the variances of V, N, and S are given as 



k=l 



t-4_r| 



(15) 



where a[,'^''^'s form the positive-half subset of An^'s. 



IV. Application to ISI Channels and Numerical 
Examples 

A. The ISI Channel and MMSE-DFE 

Fig. [3] shows the discrete-time equivalent system model of 
the finite-ISI channel with the infinite-length feedforward filter 
of the unbiased MMSE-DFE preceded by the matched filter 
(MF) for the channel. The discrete-time MF output of Fig. [3] 
is identical to the baud-rate sampled output of the continuous- 
time MF applied to the continuous-time channel, under the 
assumption that the channel is strictly limited to the Nyquist 
band. 

We also assume that the receiver knows the /^-transform 
of the finite-ISI channel response, h{D), Xk is an i.u.d. input 
sequence and Wk is additive white Gaussian noise (AWGN) 
with variance cr^ = A^q. Furthermore, Vk is the channel 
output sequence, Zk is the output sequence of the infinite- 
length MMSE-DFE feedforward filter and yk is the unbiased 
MMSE-DFE output after ideal postcursor ISI cancellation. 

Denoting X ~ xq, Xk — Xk, and Y — yo, the output of the 
the unbiased MMSE-DFE with ideal feedback |(15J is 



Y = X + J2 d-kXk + N = X + S + N = X + V 

k=l 

where N is the Gaussian noise sample observed at the DEE 
forward filter output and d-kXk is the precursor ISI sequence. 
Note we are assuming stationary random processes. It is well- 
known that the D-transform of the precursor ISI taps d-k is 
given by lITSl 

iVo 1 



d{D) = 



Po~Nn 



1 - 



(16) 



where Pq is such that logPo — 57 /_!^^ log -Rss (e ^^)d9 and 
g*{D~*) is obtained from spectral factorization: Rss{D) — 
PxRhh{D) + Ao = Pog{D)g*{D-*) with = 
h{D)h* (D^*). Notice that a convenient numerical spectral 
factorization algorithm exists for recursively computing the 
coefficients of g*[D-*) Eg), (nl. 



^2 



PxNq 
Pa -No 
PxPqNq 

27r (Po - No f J-n Rhhie-^") + Nq/Px 



Rhh{e~ 



de 



We can obtain |p|max by the absolute summation of the inverse 
D-transform of d{D) if the feedforward filter of MMSE- 
DFE is stable, i.e., M-fc| < us first consider 
the case where d{D) has P multiple first-order poles, pi for 
i = 1,2, . . . , P. Then, |p|max can be obtained by the partial 
fraction method since d{D) is a rational function. In other 
words, the inverse D-transform of individual fraction terms 
can be found and then added together to form d^k- Denoting 
a{D) = g*(D-') = EiLi i-p-D-^ ^ *^ sequence a_fc is given 
as a_fe = c^p'l■ Therefore, 




(PO-A^O) \t[^-\P^\ 



(17) 



The upper bound of |p|max can be also tightened by identifying 
the first K dominant taps: 




(i^o - No) 
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h(D) = T'^-(l + D) 
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Fig. 4: First 20 Precursor taps after unbiased MMSE-DFE at 
SNR=10 dB for four example channels. 



For the case of the multiple-order poles of d{D), the upper 
bound of |p|max can be also obtained in a similar way using 
the triangle inequality |a + 5| < \a\ + \b\. 

The SIR or the i.u.d. capacity (bits/channel use) for any 
finite-ISI channel corrupted by Gaussian noise is given lITSl 
as 



SIR= lim 

Af-j-oc 



2N- 



> lim 



1 



>l{xQ;zo\{xk}_lo) 
= I{X-Y) 

where {ufclj^J = K, fc = TVi, A^i + 1, 



(19) 

(20) 
(21) 



, A^2}- The inequal- 
ity in ( fT9] l holds due to the data processing theorem (equality 
holds if the MMSE-DFE feedforward filter is invertible). The 
inequality of ( |20] i can be obtained by applying the chain 
rule of mutual information and assuming stationarity IfTTl . 
The equality ( l2Tl i is valid because known post-cursor ISI can 
simply be subtracted out without affecting capacity. 

B. Numerical Results 

Now, let us examine the particular ISI channels, h{D) — 
2-1/2(1 + D), h[D) = 2-1(1 + D-D^ -D^) and h{D) = 
0.19 + 0.35D + 0A6D^ + 0.5D^ + 0A6D^ + 0.35D'^ + 0.19D^, 
which are well-known and previously investigated in ifTOl . 
Ifm. and h{D) = 1.6099-1/2 j^w^ + (j _ 5)2^^ ^^^^.^ 

was considered in The first 20 precursor ISI tap values are 
computed and shown in Fig. |4] for these example channels. 
In addition, we consider a complex-valued partial response 
channel: h{D) = 2''^ {{1 + j) + {1 - j)D}. The channel 
inputs are binary, except the complex-valued channel for which 
the inputs are assumed quaternary. 

Since the infinite-length MMSE-DFE is used, i.e., L — 00, 
the probability distribution of p is not available generally. 
Hence the lower bounds Cli,m = log 2 — F^/ and Cl2,m — 
log 2— F^/ along with Cslc — log 2~Fslc are considered as 





SIR 




c 

SLC 












(a) 



















-e-f', 

















SNR (dB) 



(b) 



Fig. 5: Example channel h{D) = 2-1/^(1 + D) with BPSK 
inputs (a) SIR, SLC and the new lower bounds as functions 
of SNR (b) Upper and lower bounds of F, for different M, 
less FsLC, plotted against SNR. 



functions of SNR = Px /Nq for different values of M. When 
no clustering is used, we set M — 0. In computing |p|max (and 
thus I /i I max), which was needed to calculate F^/ or F^/, we 
were able to run numerical recursive spectral factorization to 
find all non-negligible d-k coefficients relatively quickly for 
all channels considered, without resorting to the bounds of 
([TtI i or (fTSl l. We observed that the lower bounds, Cli,m and 
Cl2.m, produced similar results, so only Cli.m were chosen 
and plotted as Cl,m in Figs. |5] through |9] The SIR of each 
channel is also obtained using the simulation-based approach 

10, 0, El. 

For each capacity figure, we first plotted the SIR and Cslc- 
We then plotted Cl for M — and then another Cl by 
choosing an M value for which the Cl bound is almost as 
tight as the Cslc conjecture (this is why the Cslc curve is 
almost overwritten and indistinguishable in some figures). We 
also show for each channel how the upper and lower bounds 
of F close on each other as M increases. The bounds on 
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Fig. 6: Example channel h{D) = 2-^{l + D - D"^ - D^) 
with BPSK inputs (a) SIR, SLC and the new lower bounds 
as functions of SNR (b) Upper and lower bounds of F, for 
different M, less Fslc, plotted against SNR. 



iJ 0.005 




-15 -10 -5 5 10 15 20 

SNR (dB) 

(b) 

Fig. 7: Example channel h{D) = 0.19 + 0.35D + 0.461)2 + 
0.5D3 + 0.46134 + 0.351)5 ^ 0.191)6 with BPSK inputs (a) 
SIR, SLC and the new lower bounds as functions of SNR (b) 
Upper and lower bounds of F, for different M, less Fslc, 
plotted against SNR. 



F are shown with Fslc subtracted from them in the part 
(b) figure. In this way, it should be clear that for those SNR 
values where i^" — Fslc becomes less than zero eventually, 
F is less than Fslc, guaranteeing that r{X : Y) ~ log2 — F 
is larger than Cslc- In fact, it can be seen from the part (b) 
plots of Figs. |5] through |8] that this is true for the high SNR 
range corresponding to all rates higher than roughly 0.6 in each 
of the first four channels considered, although the difference 
F — Fslc is small (note the very small scale of the vertical 
axis in each part-b figure). In the fifth example shown in Fig. 
[9] p-^ Fslc is actually less than zero at all SNR values, 
meaning that for this channel our bound is tighter than the 
SLC in the entire SNR range. 

A significant implication here is that whenever I'{X : Y) = 
log 2 — F is larger than Cslc, there is an assurance that the 
SLC holds true. The curves of F'- - Fslc for different M 
values also provide a detailed picture of how large AI should 
be in order for Cl to get close enough to the Cslc- 



Note that the computational load for evaluating the integral 
of (O and ^ to obtain the bound depends exponentially on 
AI, the number of clusters in the pdf /y (t). The computational 
load in computing the dominant precursor ISI taps and their 
magnitude sum is minimal. The results summarized in the 
figures indicate that in each channel considered, a relatively 
small value of AI (and thus a reasonably low computational 
load) yields a bound as tight as the SLC. As a case in point, 
comparison of Fig. [8] with the results of [6| (Figure 6 of 
|6|, specifically) gives a good idea on the usefulness of an 
easily computable bound such as the one presented here. 
At a rate 0.9, for example, one can observe from a close 
examination of Fig. 6 of |[6| that the lower bound of ||6l 
approaches the SIR within about 0.88 dB with 2 iterations, 
which would require basically running the BCJR algorithm 
twice on a reduced channel trellis of 64 states. In contrast, 
our bound based on just two clusters is about 0.84 dB away 
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Fig. 8: Example channel h{D) = 1.6099-1/2 ^J^g £'7(1 + 
{i - 5)2) with BPSK inputs (a) SIR, SLC and the new lower 
bounds as functions of SNR (b) Upper and lower bounds of 
F, for different M, less Fslc, plotted against SNR. 




Fig. 9: Example channel h{D) = 2'^ {(1 + j) + {l-j)D} 
with QPSK inputs (a) SIR, SLC and the new lower bounds 
as functions of SNR (b) Upper and lower bounds of F, for 
different M, less Fslc, plotted against SNR. 



from the SIR at the same rate, as estimated from Fig. [8] This 
bound requires computation of 2^ =4 single-dimensional 
integrals, the complexity of which amounts to virtually nothing 
relative to that associated with two BCJR simulation runs in 
the method of ||6|. The simulation-based bound of [6| does 
narrow the gap to about 0.65 dB with five iterations, but at 
the expense of much more computational time. 

We stress that the value of the simulation-based SIR es- 
timation methods is not in their ability to provide easily 
obtained bounds; rather they play a critical role in estimating 
the SIR (or capacity) with a very high accuracy, given the 
ample computational resources. As for providing convenient 
and easily computed SIR estimates or bounds, the need for 
analytically evaluated bounds such as the one developed in this 
paper continues to be high. In particular, the question remains 
as to how does the proposed method perform on very long ISI 
channels with no dominant taps. A good example of this type 
of channel is the indoor wireless channel in a highly scattered 



multi-path setting. Unfortunately, our analysis indicates the 
lower bounds developed here are not very effective in this 
case, with their gaps to the SIR bigger than that of the SLC 
when computational loads are kept at reasonable levels. Easily- 
computed tight bounds for this type of channel remain elusive. 

V. Conclusion 

In this paper, we derived a lower bound to the SIR of 
the ISI channel driven by discrete and finite-amplitude inputs. 
The approach taken was to introduce a "mismatched" mutual 
information function that acts as a lower bound to the symmet- 
ric information rate between the channel input and the ideal- 
feedback MMSE-DFE filter output. This function turns out to 
be tighter than the Shamai-Laroia conjecture for a practically 
significant range of SNR values for some example channels. 
We then further lower-bounded this function by another func- 
tion that can be evaluated via numerical integration with a 
small computational load. The final computation also requires 



TO APPEAR IN IEEE INFORMATION THEORY 



10 



finding a few large precursor ISI tap values as well as the 
absolute sum of the remaining ISI terms, which can be done 
easily. The final lower bounds are demonstrated for a number 
of well-known finite-ISI channels, and the results indicate that 
the new bounds computed at a fairly low computational load 
are as tight as the SLC. 

Appendix A 
Proof of Lemma[T] 

We show below that /'(X; Y) < I{X; Y). Start by writing 

IiX;Y) -I'{X;Y) 

= {H{Y)-H'{Y))~{H{V)-H'{V)) 

fv{t)\og(i^]dt 



fait), 

= -Di^frmfzit)) + D(^fv{t)\\fG{t)) (22) 

where D(^p{t)\\q{t)j is the KuUback-Leibler (K-L) divergence 
defined as 



D 



p{t) log 



dt. 



The K-L divergence is always greater than or equal to 
zero and convex in pair {p{t)\\q{t)) [19], i.e., assuming 
Pi(i),P2(i), (7i(i), and 52 (i) are all pdfs, for < A < 1, 
we have 



D 



(\p^{t) + (1 - \)p2{t)\\\qi{t) + (1 - A)g2(<)) 

<\D(p^{t)\\qi{t))+{l-X)D(p2{t)\\q2{t)). (23) 



For the sake of clarity, we assume that X is from the binary 
phase shift keying (BPSK) alphabet, i.e., X e {±^/P^}. 
Then, 



fz{t) = \ {fait 



^Px) 



fv{t 
fait- 



Substituting pi{t) = fv{t - VP^), P2{t) = fvit + VP^), 
qi{t) - fait - VP^), q2{t) = fait + and A = 0.5 in 

(|23] |, we get 

D^fYmizit)) < \{D[fv{t-VP^)\\fG{t-VP^)) 
+D(^fv{t + v^)\\fG(t + v/P^)) } 

^D(fvmfG{t)^ 



Accordingly, ( |22] | is always greater than or equal to zero 
or r{X;Y) < IiX;Y). While this proof is for the binary 
alphabet, it is easy to see that the application of the pair- 
wise convexity of (|23] | for any i.u.d. input leads to the same 
conclusion. 



Appendix B 
Derivation of Proposition[T] 

From the pdfs of RVs V and G, we can write 



H'{V). 



fv(t) hgfaim 

r-oo ,2 



1 

-\og{2^ol)+ fy{t)dt. (24) 



Moreover, we have 

h{t) - \ {!v{t - /P^) + hit + /p^)} 

2 ^ 




We can write — \og]zit) in two different ways: 

- log ]z it) - log 2 + i log (2^a^) + ~2^^' 
-log|l + cxp(^^) 



:log2 + -log(2w2.) + 



(t + vp^y 



2al 



log s 1 + exp 



2JP^t 



Thus, we have 



/oo 
fv{t- VP^)\ogfz{t)dt 
-00 



' —00 

1 



log2 + -log(2^4] 



24 



fvit-VPx)dt 



log <; 1 + exp ( -^^^ ] } fvit - VPx)dt 



^ /"OO 

~2 



log2 + - log (2^(7^] 



log <^ 1 + exp 



2 J -00 
2VP7t - 2Px 



fvit)dt 



fvit)dt. 
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Similarly, 



Now write I'{X; Y) — log 2 — F with the new definition 



fv{t + VP^) log fz{t)dt 



^<ilog2 + ^log (27r4) 



1 

~2 



— CX3 

oo 



2al 



fv{t+VPx)dt 



log <j 1 + exp ( ) fv{t + VPx)dt 



If 1 1 1 /"^ <2 

2{log2+2l°g(2-4)j + 2y_2^/v(t^ 



Accordingly, 



log 1 + exp 



2VP^t - 2Px 



.fv{t)dt. 



I fy{t) log <i 1 + exp [ 1 !> 



^"^7 72 

X log <l + exp 



1 / (i-m,)2 

: exp ' 



-2^t - 2Px 



2< 



dt 



j=i 



oo -rV2 



/2^ 



r /-2x/PF(TaAr+m,)-2Px\1 , 
X log 1 1 + exp 1^ i — -2 ^ j I 

•^"^ /-oo „-r^/2 , , 



i=l 
2 



2-^ E [log {l + e-2«''-e-2^^--2«}] (27fl) 



— CX3 
CX3 



l0g/z(i)cii 



1 



!v{t^^JPx)\o%jz{t)dt 

oo ,2 



= log2 + -log (27ra^) 



log < 1 + exp 



2 

log <^ 1 + exp 



^h{t)dt 

oo 

-2/P^t - 2Px 



'V 



2^/P^< - 2Px 



fv{t)dt 



1 



log2 + ^log (27r4) 



-oo 2cr^ 



2 



log ^ 1 + exp ( JblI^_EA^ I 

(25) 



where the third equality is obtained with a variable change 
{t - mi)/<TN T and pi ^ mj/ ^/P^, R ^ Px/cry, and 
— cTAr/fTy. The expression (127 all can also be written as 



F = 2-^f2^r [log { 1 + e-2flp. g-20yflr-2fl | 



2-^ E E,[log{l + e-2fl''^e-20v^--2fi| 
+ log{l + e2«''?e-2'^^^-2«} 

2^-1 

E Er[log{l+ ( 
:2-(i-i) ^ Er ^log{l + 2cosh(2i?p+)e-2^^^-2« 



-2fl 



fc=l 



(27/7) 



The last equality in ( |25] l holds because is an even 

function. Finally, from ( l24l l and ( |25] l. we arrive at 



where pj's is the positive-half subset of pi's. 



I'{X;Y) 

= H'(Y) - H'{V) 
= log2 

-/ /y(t)log|l + exp 



-2^t-2Px \ 



dt. 



Appendix C 
Derivation of the Simple Bounds 



Due to the convexity of the function. 



(26) ilog(l + 2cosh(2i?p+)e-2^^^ + e-'*'^^^-''« 



in 
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p+, the upper bound of F can be found as 

F = 2-(^-i) ^ ^ log {l + 2 cosh (2i?p+)e-2^^- 



By using the convexity of the function. 



fe=i 



2R 



+e 



2^-1 

<2-(^-l) J2 |T(|p|n.ax,( 
fe=l 



log (l + e-2flPg-20VflT-2_R 

IS also found as 

2^ 

F = 2-^ ^E, [log {l + e-2«P.e-2<^^^--2flJ 

1=1 

log 1 1 + exp (~ 2R (^2^^ ^ p. 



in p, the lower bound of 



>E, 



= r(|pUax,e) 

= r(|p|,nax,^) 
<T(|p|,„ax,e) 



xe 



-20\/flr-2fi 



e=|p|„ 



(28) 



= E, 



log{l + e-20vT?r-2i^| 
i log {l + 2e-2*^-2fl + g-40yi?r-4J?| 



where, for a given |p|niax> T{\p\aiax,()) represents a straight 
line passing through the two points of E^ 5 log (l + 

2cosh(2i?6l)e^20VHr g-40\/Kr-4fljl : at 6* = and at 61 = 

IpUax. Also, IpL,^ ^ 2-^E^iIaI = 2-(^-l)E^=;^^ 
The last inequality in ( |28] l is obtained from the Cauchy- 
Schwarz inequality: |p|avg < Cp- 

Another upper bound of F can be also found as 



-2tp\/RT-2R 



r 1 

^^2-(L-i) J2 EJ-log{l + 2afc 

k=l L 
_^g-40Vi?r-4i?|| 

ilog|l + 2('2-(^-i) ^ afc]e-2^^--2« 



-40yflr-4fl 



= E, 



i log {l + 2aavge-2*^^-2^ + 



(29) 



where at — cosh{2Rp^) and aavg — 2 Sfe=i '^k — 

2-(L-i)^2^^ (,osh(2i?p^). The inequality comes from the 



concavity of Et- 



in 
last 



"i log (l + 2ae-2^v^- + e-40v^r-4fl 
a. Moreover, since it is an increasing function of a, the 
expression of ( |29l ) can be further upper-bounded by replacing 
ttavg with a' > Qfavg ■ For example, note 



aavg<2-(^-i)5:(.p+ + l) 
fe=l 

= s|plavg + l<5^P + l=«' 

where s = (cosh(2i?|p|inax) - 1) /|p|max, the slope of a 
straight line connecting two points of the convex function 

cosh(2i?p), (0,1) and (|p|max, cosh(2i?|p|max))- This gives 



F<Er 



ilog{l + 2(.sap + l)e-20vT?r-2fl 



t-4_r| 



A pu2 



(30) 



(31) 



Appendix D 
Derivation of the Tightened Bounds 

The tightened bounds are derived in a similar way using 
the convexity or concavity of the function except the cluster 
identification needs be incorporated. Since pfc = A„ + /i;, we 
can rewrite F as 



F = 2"*'^ 2^(^-^^) 

n=l ^ 

^ E, [log {l + e-2H(A..+A„)g-20ySr-2fi| 

i=l 

2 A-/ 

n=l ^ 

^ E^ - log 1 1 + 2 cosh (2i?/x^ 



1=1 



4) 



^g-20Vflr-2fi ^ g-4i?,A„g-4(/>V^r-4J?,| 



<2-*^^ ('2-(^-*^-i) ^ |r„(| 

2 A-/ 

= 2-*^^|r„(|MUax,0) 

2 A-/ 

= 2-^^^(r„(|/.Uax,0) 



Mimax, 0) 



e=p/ 



2-(-L-Af-l)^2i-*^-l^+ 



n=l 
2" 



'=IpU 



<2 



-M 



n=l 



A pul 



(32) 



where /i/^'s form the positive-half subset of pi's and, 
for a given |/i|i„ax, Tn(|Ai|max, 6*) is a straight line 
that passes through the convex function E,- i log 1 1 + 

2cOsh(2i?0)e-2«A„g-20yflr-2fl ^ g-4flA„ g-40ySr-4/? 1 



at 61 



and 

-,L-M 



2-(L-M)^2^- l^^l ^ 2- 



I Ml max- 

(L-Af-1) 



Moreover, |/^|avg 
X)i=i ■ The last 
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inequality also follows from |/i|avg < fju> and note = cation is obtained as 

\/^p ~ iMlmax = |/f|max " |A|max- 

Another form of tightened upper bound of F is obtained as 



-(L-M-l) 



=1 

xe" 

2" 



n=l ^ 

^ i log { 1 + 2 cosh (2i?iU+) 

n=l ^ 

2L-M-1 

^ EJilog{l + 2Ae-2«^"e-2^^--2^ 
;=i L 

_|_g-4iiA„g-4<^ViiT-4_R| ^ 

<2-^VEjilog(l + 2(2-(^-^-i) V a) 

n=l ^ 1=1 ^ 

^^-2R\„^-24>VRt-2R _|_ g-4iJA„g-4<;i\/Rr-4K| 

= 2-^ 5^ E, ^ { ^ + 2/3avge-'«^"e-2^^--2« 

_|_g-4ii;A„ g-40Viir-4ii| 

2M 

< 2-^ V E, [i log 1 1 + 2/3'e-2^^"e-2^^-2« 

n=l '- 
_|_g-4flAn g-40\/Rr-4ii| 

2" p 

= 2-^5^E, -\og[l + 2{sMC7^ + l)e 

n=l L 

} 



-2iiAn 



^g-20VflT-2_R _|_ g-4_RA„g-40VflT-4iil 



A p«2 



(33) 



where A ^ cosh (2i?A*i+), Avg = 2-(^-^-i) E'=r""' A 
2-(^-^-i)X;f=i'"~'cosh(2i?/i+) and 



-"avg 



<2-(^-M-l) ^ (,SMM^ + 1) 



SM l/^la^g + 1 < SMO-y:, + 1 = /3' 



which is based on a straight hne connecting 
two points of the convex function cosh(2i?y^), 
(0,1) and (|/;i|max,cosh(2i?|/x|max), having a slope 

Sm = (C0sh(2ii|ju|inax) - 1) /|At|max. 

The tightened lower bound of F based on cluster identifi- 



F = 2-^^ I 2-iL-M) 

n=l ^ 

2L-M 

^ E, [log {1 + e-2«(M.+A.)e-2<Av^-2ij|j^ 

i=l ^ 

2M 

n=l ^ 

^ E, [log {1 + e-2«/^.e-2«^-e-2^^-2«}]') 

i=l ^ 

> 2-^ ^ E Jlog |l + cxp (- 2i?(2-(^-^) Y: . 

n=l LI \ V 
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where Aj^'s form the positive-half subset of A„'s. 
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